1. Field of the Invention
The present invention relates to acoustic processing technology, and more particularly to a three-dimensional acoustic processor which provides a three-dimensional acoustic effect to a listener in a reproducing sound field via a headphone or the like.
2. Description of Related Art
In general, to achieve accurate reproduction or location of a sound image, it is necessary to obtain the acoustic characteristics of the original sound field up to the listener and the acoustic characteristics of the reproducing sound field from the acoustic output device, such as a speaker or a headphone, to the listener. In an actual reproducing sound field, the former acoustic characteristics are added to the sound source and the latter characteristics are removed from the sound source, so that even using a speaker or a headphone it is possible to reproduce to the listener the sound image of the original sound image of the original sound field, or so that it is possible to accurately localize the position of the original sound image.
In the past, in order to add the acoustic characteristics from the sound source to the listener of the original sound field and remove the acoustic characteristics of the reproducing sound field from the acoustic output device such as a speaker or a headphone up to the listener, a FIR (finite impulse response, non-recursive) filter having coefficients that are the impulse responses of each of the acoustic spatial paths was used as a filter to emulate the transfer characteristics of the acoustic spatial path and the reverse of the acoustic characteristics of the reproducing sound field up to the listener.
However, when measuring the impulse response in a normal room for the purpose of obtaining the coefficients of an FIR filter in the past, the number of taps of the FIR which represent those characteristics when using an audio-signal sampling frequency of 44.1 kHz is several thousand or even greater. Even in the case of the inverse of the transfer characteristics of a headphone, the number of taps required is several hundred or even greater.
Therefore, when using FIR filters, there is a huge number of taps and computation required, causing the problems that in an actual circuit implementation it is necessary to have a plurality of parallel DSPs or convolution processors, this hindering a reduction in cost and the achievement of a physically compact circuit.
In addition, in the case of localizing the sound image, it is necessary to perform parallel processing of a plurality of channel filters for each of the sound image positions, making it even more difficult to solve the above-noted problems.
Additionally, in an image-processing apparatus which processes images which have accompanying sound images, such as in real-time computer graphics, the amount of image processing is extremely great, so that if the capacity of the image-processing apparatus is small or many images must be processed simultaneously, the insufficient processing capacity produces cases in which it is not possible to display a continuous image, and the image appears as a jump-frame image. In such cases, there is the problem that the movement of the sound image, which is synchronized to the movement of the visual image, becomes discontinuous. In addition, in cases in which the environment is different from the expected visual/auditory environment of, for example, the user""s position, there is the problem of the apparent movement of the visual image being different from the movement of the sound image.
In consideration of the above-noted drawbacks of the prior art, an object of the present invention is to perform linear predictive analysis of the impulse response which represents the acoustic characteristics to be added to the original signal for the purpose of adding characteristics to the acoustic characteristics, the linear predictive coefficients being used to form a synthesis filter, thereby greatly reducing the number of filter taps, so as to achieve such effects as reduction in size and cost of the related hardware, and an increase in the processing speed achieved thereby. In the case of performing the above-noted linear predictive analysis and using a filter of lower order than the original number of impulse response samples to approximate the frequency characteristics, a three-dimensional acoustic processor is provided in which in particular in the case of high complexity in which the sharp peaks and valleys existing in the original impulse response frequency characteristics, in order to prevent a loss of approximation accuracy, before the linear predictive analysis is performed, to eliminate any auditory change the frequency characteristics of the original impulse responses are smoothed and compensated in the frequency domain, thereby approaching the original impulse response frequency characteristics and enabling a reduction of the number of filters without causing a change in the overall acoustic characteristics.
Another object of the present invention is to provide a three-dimensional acoustic processor in which the acoustic characteristics from a plurality of positions from which a sound image is to be localized are divided into characteristics common to each position and individual characteristics for each position, the filters which add these being disposed in series to control the position of the sound image, thereby reducing the amount of processing performed. In the case in which the sound image is caused to move, by localizing a single sound image at a plurality of locations and controlling the difference in acoustic output level between the different locations, the sound image is smoothed therebetween, interpolation being performed between the positions of the visual image which moves discontinuously, thereby achieving moving of the sound image which matches the thus interpolated positions. In addition, a three-dimensional acoustic processor is provided wherein, in the case in which a reproducing sound image is reproduced using a DSP (digital signal processor) or like, to avoid complexity of registers and like, and to perform the desired sound image localization, localization processing is performed for only the required virtual sound source.
According to the present invention, a three-dimensional acoustic processor is provided which localize a sound image using a virtual sound source, wherein the acoustic characteristics to be added to the sound signal are formed by a linear synthesis filter having filter coefficients that are the linear predictive coefficients obtained by linear predictive analysis of the impulse response which represents those acoustic characteristics, the desired acoustic characteristics being added to the above-noted original signal via the above-noted linear synthesis filter.
The above-noted linear synthesis filter includes a short-term synthesis filter having an IIR filter configuration and which uses the above-noted linear predictive coefficients which adds the desired frequency characteristics to the above-noted original signal, and a pitch synthesis filter having an IIR filter configuration and which uses the above-noted linear predictive coefficient which adds the desired frequency characteristics to the above-noted original signal. The above-noted pitch synthesis filter is formed by a pitch synthesis section with regard to direct sounds with a large attenuation factor, a pitch synthesis section with regard to reflected sounds with a small attenuation factor, and a delay section which applies a delay time thereto. Furthermore, the inverse acoustic characteristics of an acoustic output device such as a headphone or a speaker are formed by means of a linear predictive filter having filter coefficients which are the linear predictive coefficients obtained by linear predictive analysis of the impulse response which represents the acoustic characteristics thereof, the acoustic characteristics of the above-noted acoustic output device being eliminated via this filter. The above-noted linear predictive filter is formed as an FIR filter which uses the above-noted linear predictive coefficients.
According to the present invention, a three-dimensional acoustic processor which uses linear prediction is provided, wherein the desired acoustic characteristics to be added to the original signal are formed by a linear synthesis filter having filter coefficients that are the linear predictive coefficients obtained by means of linear predictive analysis of the impulse response which represents those acoustic characteristics, these desired acoustic characteristics being added to the above-noted original signal via this filter, the power spectrum of the desired impulse response representing the above-noted acoustic characteristics being divided into a plurality of critical frequency bands, the above-noted linear predictive analysis being performed based on impulse signals determined from the power spectrum which is used to represent the signal sounds within each of the critical bands, thereby determining the filter coefficients of the above-noted linear synthesis filter.
The spectral signals which represents the signal sounds within each critical band are taken as the accumulated sums, maximum values, or average values of the power spectrum within each critical band. Interpolation is performed between the power spectrum signals which represent the signal sounds within each of the above-noted critical bands, and the filter coefficients of the above-noted linear synthesis filter are determined by performing the above-noted linear predictive analysis based on the impulse signal determined from the above-noted output interpolated signal. For the above-noted interpolation, first order linear interpolation or high-order Taylor series interpolation are used. In addition, an impulse response which indicates the acoustic characteristics for the case of a series linking of the propagation path in the original sound field and the propagation path having the inverse acoustic characteristics of the reproducing sound field is used as the impulse response indicating the above-noted sound field, a filter to which is added the acoustic characteristics of the original sound field and a filter which eliminates the acoustic characteristics in the reproducing sound field being linked as one filter and used as the above-noted linear synthesis filter for determination of the linear predictive coefficients based on the above-noted linked impulse response. A compensation filter is used to reduced the error between the impulse response of the linear synthesis filter which uses the above-noted linear predictive coefficients and the impulse response which indicates the above-noted acoustic characteristics.
A three-dimensional acoustic processor according to the present invention which localizes a sound image using a virtual sound source has a first acoustic characteristics adding filter which is formed by a linear synthesis filter which has filter coefficients that are the linear predictive coefficients obtained by linear predictive analysis of the impulse response which represents each of the acoustic characteristics of one or each of a plurality of propagation paths to the left ear to be added to the original signal, a first acoustic characteristics elimination filter which is connected in series with the above-noted first acoustic characteristics adding filter, and which is formed by a linear predictive filter having filter coefficients which represent the inverse of acoustic characteristics for the purpose of eliminating the acoustic characteristics of an acoustic output device to the left ear, these filter coefficients being obtained by a linear predictive analysis of the impulse response representing the acoustic characteristics of the above-noted acoustic output device, a second acoustic characteristics adding filter which is formed by a linear synthesis filter which has filter coefficients that are the linear predictive coefficients obtained by a linear predictive analysis of the impulse response which represents each of the acoustic characteristics of one or each of a plurality of propagation paths to the right ear to be added to the original signal, a second acoustic characteristics elimination filter which is connected in series with the above-noted second acoustic characteristics adding filter, and which is formed by a linear predictive filter having filter coefficients which represent the inverse of acoustic characteristics for the purpose of eliminating the acoustic characteristics of an acoustic output device to the right ear, these filter coefficients being obtained by a linear predictive analysis of the impulse response representing the acoustic characteristics of the above-noted acoustic output device, and a selection setting section which selectively sets the parameters for the above-noted first acoustic characteristics adding filter and above-noted second acoustic characteristics adding filter responsive to position information of the sound image.
The above-noted first and second acoustic characteristics adding filters are configured from a common section which adds characteristics which are common to each of the acoustic characteristics of the acoustic path, and an individual characteristic section which adds characteristics individual to each of the acoustic characteristics of each acoustic path. In addition, there is a storage medium into which is stored the calculation results for the above-noted common section of the desired sound source, and a readout/indication section which reads out the above-noted stored calculation results, the readout/indication section directly to the above-noted individual characteristic section the read out calculation results, by means of the readout it performs. In addition to storing the above-noted calculation results of the common section for the desired sound source, the storage medium can also store the calculation results of the corresponding first or second acoustic characteristics elimination filter.
The above-noted first acoustic characteristics adding filter and second acoustic characteristics adding filter further have a delay section which imparts a delay time between the two ears, so that by making the delay time of the delay section of either the first or the second acoustic characteristics adding filter the reference (zero delay time), it is possible to eliminate the delay section which has this delay of zero. The above-noted first acoustic characteristics adding filter and second acoustic characteristics adding filter each further have an amplification section which enables variable setting of the output signal level thereof, the above-noted selection setting section relatively varying the output signal levels of the first and the second acoustic characteristics adding filters by setting the gain of these amplification sections in response to position information of the sound image, thereby enabling movement of the localized position of the sound image. The above-noted first and second acoustic characteristics adding filters can be left-to-right symmetrical about the center of the front of the listener, in which case, the parameters for the above-noted delay sections and amplification sections are shared in common between positions which correspond in this left-to-right symmetry.
In accordance with the present invention, the above-noted three-dimensional acoustic processor has a position information interpolation section which interpolates intermediate position information from past and future sound image position information, interpolated position information from this position information interpolation section being given to the selection setting section as position information. In the same manner, there is a position information prediction section which performs predictive interpolation of future position information from past and current sound image position information, the future position information from this position information prediction section being given to the selection setting section as position information.
The above-noted position information prediction section further includes a regularity judgment section which performs a judgment with regard to the existence of regularity with regard to the movement direction, based on past and current sound image position information, and in the case in which the regularity judgment section judges that regularity exists, the above-noted position information prediction section provides the above-noted future position information. It is possible to use the visual image position information from image display information for a visual image which generates a sound image in place of the above-noted sound image position information. So that the above-noted selection setting section can further provide and maintain a good audible environment for the listener, it can move the above-noted environment in response to position information given with regard to the listener.
In accordance with the present invention, a three-dimensional acoustic processor is provided which localizes a sound image by level control from a plurality of virtual sound sources, this processor having an acoustic characteristics adding filter which adds the impulse response which indicates the acoustic characteristics of each of the above-noted virtual sound sources to the listener and which is given with respect to two adjacent virtual sound sources between which is localized a sound image, this acoustic characteristics adding filter storing filter calculation parameters for the two adjacent virtual sound sources, and when one of the two adjacent virtual sound sources are moved to an adjacent region, without changing the acoustic characteristics filter calculation parameter corresponding to that virtual sound source, the acoustic characteristics filter calculation parameters of the other virtual sound source are updated to the virtual sound source which exists in the adjacent region.
According to the present invention, a linear synthesis filter is formed which has linear predictive coefficients that are obtained by linear predictive analysis of the impulse response which represents the desired acoustic characteristics to be added to the original signal. Then compensation is performed of the linear predictive coefficients so that the time-domain envelope (time characteristics) and the spectrum (frequency characteristics) of this linear synthesis filter are the same as or close to the original impulse response. Using this compensated linear synthesis filter, the acoustic characteristics are added to the original sound. Because the time-domain envelope and spectrum are the same as or close to the original impulse response, by using this linear synthesis filter it is possible to add acoustic characteristics which are the same as or close to the desired characteristics. In this case, by making the linear synthesis filter a pitch filter and a short-term filter which are IIR filters (recursive filters), it is possible to form the linear synthesis filters with a great reduction in the number of filter taps as compared with the past. In this case, the above-noted pitch synthesis filter is used to control the time-domain envelope and the short-term synthesis filter is mainly used to control the spectrum.
According to the present invention, the acoustic characteristics are changed with consideration given to the critical bandwidths in the frequency domain of the impulse response indicating the acoustic characteristics. From these results, the auto-correlation is determined. In the case of making the change with consideration given to the above-noted critical bandwidth, because the human auditory response is not sensitive to a shift in phase, it is not necessary to consider the phase spectrum. By smoothing the original impulse response so that there is no auditory perceived change, consideration being given to the critical bandwidth, it is possible to achieve a highly accurate approximation of frequency characteristics using linear predictive coefficients of low order.
According to the present invention, filters are configured by dividing the acoustic characteristics to be added to the input signal into characteristics which are common to each position at which the sound image is to be localized and individual characteristics. In the case of adding acoustic characteristics, these filters are connected in series. By doing this, it is possible to reduce the overall amount of calculations performed. In this case, the larger the number of individual characteristics, the larger will be the effect of the above-noted reduction in the amount of calculations. By storing the results of the processing for the above-noted common parts beforehand onto a storage medium such as a hard disk, for applications such as games, in which the sounds to be used are pre-established, it is possible to perform real-time processing of input of the individual acoustic characteristics to the filters for each position by merely reading out the signal directly from the storage medium. For this reason, there is not only a reduction in the amount of calculations, but also there is a reduction in the amount of storage capacity required, compared to the case of simply storing all information in the storage medium.
In addition, in addition to storing the output signal of the filter to add the common characteristics to each position, it is possible to store into the storage medium the output signals obtained by input to filters for eliminating acoustic characteristics. In this case, there is no need to perform processing of the acoustic characteristics elimination filter in real time. Thus, it is possible to use a storage medium to move a sound image with a small amount of processing.
Further, according to the present invention, it is possible to move a sound image continuously by moving the sound image in accordance with the interpolated positions of a visual image which is moving discontinuously. Also, by inputting the user""s auditory and visual environment into an image controller and a sound image controller it is possible to achieve apparent agreement between the movement of the visual image and the movement of the sound image, by using this information to control the movement of the visual image and sound image.
According to the present invention, by compensating for the waveform of the synthesis filter impulse response in the time domain, it is easy to control the difference in level between the two ears. By doing this, it is possible to reduce the number of filters without changing the overall acoustic characteristics, making a DSP implementation easier, and further it is possible to reduce the amount of required memory capacity by only performing localization processing for the required virtual sound sources for the purpose of localizing the desired sound image.